test: one source fingerprint, in Python, called by both harnesses (#907) - #911

…mmandprompt#907)

test/lib.sh and test/pytest/pgc_cluster.py each carried their own answer to
"what was this binary built from". On 2026-09-09 the pair produced four defects
between them -- two in each copy, and NOT ONE was found by whoever wrote that
copy:

    objstore/*.c never walked            python   @linuxhikerpm, commandprompt#897
    the bare NAME instead of the path    python   found while fixing the above
    `xargs -0 cat | md5sum`, no bounds   shell    @linuxhikerpm, commandprompt#898
    each build dir's Makefile omitted    python   found while writing the twin

The Python docstring asserted "the same input set as pgc_source_fingerprint in
test/lib.sh" throughout all four. It was false when written and stayed false
through two rounds of fixing. A prose claim of agreement is not a mechanism, and
it is worse than silence because it is what stops the next person checking.

Python, not shell, which is the opposite of what commandprompt#907 first proposed
--------------------------------------------------------------------
jd's constraint decided it: the single implementation belongs in the more
portable language. bash is largely a GNU thing; Python is present on FreeBSD and
Windows where bash is not. lib.sh already requires bash, so calling a more
portable interpreter from it cannot cost portability.

My argument against this direction was that lib.sh invokes python3 zero times, so
this escalates from "53 suites need it" to "every suite needs it at gate time".
That is true and it is not a cost, for the reason above. Measured, expecting to
report a subprocess penalty:

    shell, forking md5sum once per file    239 ms/call
    the module, one interpreter start       26 ms/call
    across 261 suites x 2 fingerprints      124 s  ->  13 s

The portable direction is also 9x faster. I had it backwards in both dimensions.

A fifth defect, which unifying them found
------------------------------------------
`sort -z` orders by LOCALE COLLATION, and nothing in this harness pins a locale.
The same tree fingerprinted two ways depending on whose machine it was:

    LC_ALL=C             6d122a7158d5
    LC_ALL=en_US.UTF-8   0b59bd75fa4f

en_US.UTF-8 is a common desktop default, so this is a developer stamping a tree
and CI reading it back and calling the binary stale -- a false FATAL arriving
from the environment rather than from the source. The module sorts BYTES, which
is what LC_ALL=C produced and what every stamp already on disk was written with,
so no existing stamp is invalidated. Arms in both harnesses.

Equivalence, established rather than asserted
----------------------------------------------
A differential run of the module against the shell it replaces, over trees built
to break the ways this pair has actually broken. 17 shapes, manifest AND
fingerprint compared:

    the real source tree, minimal, a recursed module, a dir with sources but no
    Makefile, collation-sensitive names, a symlinked source file, a symlinked
    build directory, no src/, an empty tree, root .control and .sql, non-source
    files, spaces and punctuation, unicode, a Makefile at depth 3, a trailing
    slash, a /./ segment, five recursed modules

    AGREE=17  DIVERGE=0

Two of those are subtle enough to be worth naming. `find -type f` tests the LINK,
so a symlinked source is not in the shell's manifest, while `pathlib.is_file()`
FOLLOWS it and would have added one; the module excludes symlinks explicitly.
And `find` does not descend a symlinked directory, so build dirs discovered
through one differ -- which is why the module canonicalises the root first.

The mechanism of two arms had to change with the implementation
----------------------------------------------------------------
The failed-digest arms in 340 and test_build_refusal.py stubbed `md5sum` on PATH.
The digest is hashlib now, which no PATH can reach, so the stub would have left
both arms GREEN while testing nothing -- the exact shape this corpus refuses.

A real read failure needs a real reader who is denied, and root is denied
nothing: chmod 000 is invisible to it. Measured before the arms were rewritten:

    as root      28a7149e07ae   <- reads the mode-000 file regardless
    as postgres  (empty)        <- the failure the arm needs

So the tree is built outside any mode-0700 directory and read by a second user,
with a premise asserting that reader agrees with a privileged one WHILE nothing
is denied -- otherwise the arm measures the user switch rather than the failure.
Where no non-root user exists it records expect.cannot_run rather than passing.

And the arm that would catch this issue recurring
--------------------------------------------------
selftest 380's static guards follow the fingerprint to its new file, plus three
new arms: neither caller may keep a private implementation, and the module may
import nothing from test/pytest/. A static assertion of ABSENCE is the shape that
most often cannot fail, so each was proved against the REAL files rather than
only against fixtures -- a fixture proves the pattern matches something, not that
the arm aimed at the real file would fire:

    pgc_cluster.py grows a private digest      HELD
    lib.sh grows a private md5sum loop         HELD
    the module imports from the pytest tree    HELD

    harness_selftest   407 passed + 0 failed + 0 unrunnable, rc=0
    pytest corpus       91 passed
    docs_style           9 checks PASSED
    shellcheck -S error  clean

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a